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In the Claims: 

The following listing of claims will replace all prior versions and/or listings of claims in 
the application: 
Listing of Claims: 

What is claimed is: 

1. (Original) A computer-readable medium comprising software to detect passback 
events, which when executed by a computer system, cause said computer system to perform 
operations comprising a method of: 

defining a passback direction for a video monitored area; 
accessing video collected from said video monitored area; 

analyzing said video collected from said video monitored area to detect passback events 
in said video monitored area based on said passback direction; and 
initiating an action based on any detected passback events. 

2. (Original) A computer-readable medium as in claim 1, wherein said passback direction 
is defined based on at least one of an image of said video monitored area and video of said video 
monitored area. 

3. (Original) A computer-readable medium as in claim 1, wherein said passback direction 
is user defined. 
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4. (Original) A computer-readable medium as in claim 3, wherein said passback direction 
is determined via a graphical user interface to identify said passback direction in at least one of 
an image of said video monitored area and video of said video monitored area. 

5. (Original) A computer-readable medium as in claim 4, wherein said graphical user 
interface is adapted to permit a user to draw said passback direction on at least one of an image 
of said video monitored area and video of said video monitored area. 

6. (Original) A computer-readable medium as in claim 1, wherein said defining said 
passback direction comprises learning said passback direction through observation of said video 
monitored area. 

7. (Original) A computer-readable medium as in claim 6, wherein said learning said 
passback direction comprises: 

learning a normal direction for said video monitored area based on said observation of 
said video monitored area; and 

determining said passback direction based on said normal direction. 

8. (Original) A computer-readable medium as in claim 6, wherein said passback direction 
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is learned as a function of time. 

9. (Original) A computer-readable medium as in claim 8, wherein said passback direction 
is learned for a first time period, and wherein said defining said passback direction fiirther 
comprises learning at least one additional passback direction for said video monitored area for at 
least one additional time period based on said observation of said video monitored area. 

10. (Original) A computer-readable medium as in claim 1, wherein said passback 
direction is defined in association with at least one of a region of interest of said video monitored 
area, a time interval of detection, and a sensitivity of detection. 

11. (Original) A computer-readable medium as in claim 1, wherein said accessing video 
comprises accessing video in real time fi*om a video camera. 

12. (Original) A computer-readable medium as in claim 1, wherein said accessing video 
comprises accessing stored video. 

13. (Original) A computer-readable medium as in claim 1, wherein said analyzing video 
comprises extracting tracks from said collected video. 
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14. (Original) A computer-readable medium as in claim 13, wherein said extracting 
tracks comprises: 

extracting foreground from said collected video to obtain extracted foreground; 
detecting trackable features based on said extracted foreground; 
tracking said trackable features based on said extracted foreground to obtain extracted 
tracks; and 

filtering said extracted tracks; and 

wherein said analyzing video further comprises detecting passback events based on said 
extracted tracks. 

15. (Original) A computer-readable medium as in claim 14, wherein said extracting 
foreground is based on pixel statistics of a current frame and at least one past frame. 

16. (Original) A computer-readable medium as in claim 14, wherein said extracting 
foreground is based on three- frame motion differencing. 

17. (Original) A computer-readable medium as in claim 14, wherein said detecting 
trackable features comprises: 

subdividing foreground of said collected video into cells; 
determining if each cell is appropriate for tracking; and 
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designating appropriate cells for tracking as trackable features. 

18. (Original) A computer-readable medium as in claim 17, wherein said determining if 
each cell is appropriate for tracking is based on at least one of texture in each cell, intensity range 
in each cell, number of different intensity values in each cell, presence of at least one edge in 
each cell, and presence of at least one comer in each cell. 

19. (Original) A computer-readable medium as in claim 14, wherein said tracking said 
trackable features comprises: 

correlating each trackable feature with previous tracked features to obtain correlated 
trackable features and correlated previous tracked features; 

comparing directions of said correlated trackable features and said correlated previous 
tracked features; 

designating said correlated trackable features as tracked features if said directions are 
consistent; and 

updating said correlated previous tracked features with said correlated trackable features 
if said directions are consistent; and 

wherein said filtering said extracted tracks comprises filtering said tracked features. 

20. (Original) A computer-readable medium as in claim 19, wherein said correlating each 
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trackable feature comprises performing two-dimensional correlation in an area predicted from 
said previous tracked features to contain said trackable feature. 

21. (Original) A computer-readable medium as in claim 19, wherein said correlating each 
trackable feature comprises performing one-dimensional correlation on a horizontal projection 
and a vertical projection of an area predicted from said previous tracked features to contain said 
trackable feature. 

22. (Original) A computer-readable medium as in claim 19, wherein said updating said 
correlated previous tracked features is based on at least one of a direction and an age of said 
correlated trackable features. 

23. (Original) A computer-readable medium as in claim 14, wherein said fihering said 
extracted tracks comprises at least one of filtering noise from said extracted tracks, filtering out 
nuisance extracted tracks from said extracted tracks, filtering out extracted tracks from said 
extracted tracks that are not sufficiently long lived, and filtering out extracted tracks from said 
extracted tracks that are inconsistent with other nearby extracted tracks. 

24. (Original) A computer-readable medium as in claim 13, wherein said extracting 
tracks comprises: 
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extracting foreground from said collected video to obtain extracted foreground; 
determining optical flow based on said extracted foreground to obtain said extracted 
tracks; and 

filtering said extracted tracks. 

25. (Original) A computer-readable medium as in claim 24, wherein said extracting 
foreground is based on pixel statistics of a current frame and at least one past frame. 

26. (Original) A computer-readable medium as in claim 24, wherein said extracting 
foreground is based on three-frame motion differencing. 

27. (Original) A computer-readable medium as in claim 24, wherein said determining 
optical flow comprises: 

determining current flow vectors for foreground pixels of said extracted foreground; 
comparing current and previous flow vectors to obtain consistent current flow vectors; 

and 

aggregating consistent current flow vectors into cumulative flow vectors. 

28. (Original) A computer-readable medium as in claim 24, wherein said filtering said 
extracted tracks comprises at least one of filtering noise from said extracted tracks, filtering out 
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nuisance extracted tracks from said extracted tracks, filtering out extracted tracks from said 
extracted tracks that are not sufficiently long lived, and filtering out extracted tracks from said 
extracted tracks that are inconsistent with other nearby extracted tracks. 

29. (Original) A computer-readable medium as in claim 1, wherein said analyzing video 
comprises 

comparing a track determined from said video to said passback direction to detect whether any 
passback event occurred. 

30. (Original) A computer-readable medium as in claim 29, wherein said comparing said 
track and said passback direction comprises: 

representing said track as a track vector; 

representing said passback direction as a passback direction vector; 

determining a dot product of said track vector and said passback direction vector; and 

comparing said dot product to a threshold to determine whether said track vector is 
substantially aligned with said passback direction; and 

when said track vector is substantially aligned with said passback direction, filtering said 
track vector by at least one of spatial filtering and temporal filtering to determine whether a 
sufficient number of track vectors are substantially aligned with said passback direction. 
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31. (Original) A computer-readable medium as in claim 1, wherein said initiating said 
action comprises generating a report for each detected passback event. 

32. (Original) A computer-readable medium as in claim 31, wherein said report 
comprises at least one of a time of each detected passback event, a date of each detected passback 
event, a direction of each detected passback event, a location of each detected passback event, a 
size of a crossing area of each detected passback event, an indication of occlusion of each 
detected passback event, at least one snapshot of each detected passback event, at least one video 
of each detected passback event, a number of detected passback events over a time interval, and a 
time histogram of detected passback events over a time interval 

32. (Original) A computer-readable medium as in claim 1, wherein said initiating said 
action comprises initiating a triggered response for each detected passback event. 

34. (Original) A computer-readable medium as in claim 33, wherein said triggered 
response comprises at least one of initiating an alarm, controlling an audible alarm system, 
controlling a silent alarm system, accessing an alerting device, accessing an alerting system, 
sending an alert, logging alert data to a database, taking a snapshot of each detected passback 
event, culling a snapshot of each detected passback event from said video, recording video of 
each detected passback event, controlling a camera to zoom in on each detected passback event, 



-10- 



Applicant: LIPTONetaL 
Appl. No. 10/766,949 

controlling a camera to track each detected passback event, performing recognition of each 
detected passback event, closing at least one door, and controlling an access control system to at 
least one of lock, unlock, open, and close at least one portal. 

35. (Original) A computer-readable medium as in claim 1, said method further 
comprising defining at least one of an option for reporting any detected passback event and an 
option for responding to any detected passback event. 

36. (Original) A computer-readable medium as in claim 1, wherein said method is 
adapted to detect passback events in real time. 

37. (Original) A method to detect passback events comprising: 
defining a passback direction for a video monitored area; 
accessing video collected from said video monitored area; 

analyzing said video collected from said video monitored area to detect any passback 
events in said video monitored area based on said passback direction; and 
initiating an action based on any detected passback events. 

38. (Original) A system to detect passback events, comprising: 

at least one video camera to monitor at least one video monitored area and to obtain video 
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of said video monitored area; 

an analysis system coupled to said video camera, said analysis system comprising a 
computer system and a computer-readable medium, said computer-readable medium comprising 
software to control said computer system according to a method, said method comprising 
analyzing said obtained video to detect passback events in said video monitored area based on a 
passback direction for said video monitored area; and 

a user interface coupled to said analysis system. 

39. (New) An apparatus to detect passback events adapted to perform operations 
comprising a method of: 

defining a passback direction for a video monitored area; 
accessing video collected fi'om said video monitored area; 

analyzing said video collected fi'om said video monitored area to detect passback events 
in said video monitored area based on said passback direction; and 
initiating an action based on any detected passback events. 

40. (New) The apparatus of claim 39, wherein said apparatus comprises application- 
specific hardware to emulate a computer and/or software adapted to perform said defining, said 
accessing, said analyzing, and said initiating. 
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